CodeByVoice: A Voice-Controlled Python Programming Assistant for Differently-Abled Users

Abstract

CodeByVoice is a programming assistant which you control with your voice; it helps you code in Python without having to touch a keyboard via natural language commands. We have a mic which captures your voice and a speech recognition engine which turns that into text. That input is then processed to produce correct Python code as per your instructions. CodeByVoice includes a range of features from generating new code, to editing what you already have, running programs and saving files with simple voice commands. Also, we have a graphical interface which displays the code which is generated, you may review it there or go in and make manual changes if you wish. By running the program, the program will capture the output to display it and have the results read aloud via a text-to-speech engine. The program will also manage files by saving the user-created programs as Python (.py) files by name. The Error handling Mechanism improves seamless interaction by recognizing user input as invalid commands or a violation of the program\'s syntax and offers guidance to remedy the issue. CodeByVoice offers the most streamlined and user-friendly environment for programming by offering the ability to code and execute code using speech, as well as receive audio feedback. This is also very useful for novice users, users who code for utility and those who prefer not to use a keyboard for coding.

Introduction

This study explores the development of CodeByVoice, a voice-controlled programming assistant designed to make programming more accessible through natural-language voice commands. Programming has become an essential skill supporting advancements in fields such as artificial intelligence, data science, and software development. However, conventional programming environments rely heavily on keyboard-based interaction, creating challenges for users who face difficulties with traditional input methods. The study investigates how speech recognition, natural language processing, and automated code generation can be integrated to create a more inclusive programming environment.

The review highlights the evolution of integrated development environments (IDEs), which have advanced from basic coding platforms to systems incorporating debugging, testing, code assistance, error detection, and productivity-enhancing tools. Existing voice-based programming approaches have mainly focused on speech-to-text conversion, code dictation, or limited command execution. Although speech recognition technologies have improved significantly, previous systems often lack complete programming capabilities, including code generation, editing, execution, debugging, and feedback through voice. Challenges such as syntax errors, ambiguity in spoken commands, and limited complexity handling have restricted the effectiveness of earlier voice programming solutions.

The literature review examines advancements in speech recognition, natural language programming, and human-computer interaction. Early systems demonstrated the feasibility of spoken programming but were limited by strict command structures and syntax requirements. Later developments, including neural-network-based speech recognition models and natural-language interfaces, improved accuracy and usability. Research on systems such as spoken programming environments and voice-assisted IDEs showed that voice interaction can support coding activities, but most solutions addressed only specific tasks rather than providing a complete programming workflow. The review identifies a research gap in creating an integrated platform that combines speech recognition, natural-language understanding, code generation, execution, and voice feedback.

The proposed work introduces CodeByVoice, a voice-activated Python programming assistant developed using a qualitative system design methodology. The system aims to allow users to create, modify, execute, and manage Python programs using spoken commands instead of traditional keyboard input. The architecture consists of several interconnected modules:

• Speech Input Module: Captures user voice commands through a microphone.
• Speech Recognition Module: Converts spoken instructions into text using modern speech-processing techniques.
• Command Interpretation Module: Uses natural language processing to identify user intentions, such as code creation, editing, execution, or file management.
• Code Generation Module: Converts interpreted commands into syntactically correct Python code using predefined rules and templates.
• Code Editor Interface: Displays generated code for user review and modification.
• Execution and Feedback Module: Runs programs and provides visual and auditory feedback through text-to-speech technology.
• File Management Module: Enables voice-based saving and organization of programs.

Python was selected as the implementation language due to its readability, popularity, and suitability for educational and professional programming environments. The proposed system combines voice interaction with visual confirmation, allowing users to verify and refine generated code while maintaining accessibility.

The experimental evaluation of CodeByVoice demonstrated that voice-based programming can function as a practical alternative interaction method. The prototype successfully captured voice commands, converted them into text, interpreted programming intentions, generated Python code, executed programs, and provided both visual and audio feedback. These results indicate that integrating speech recognition, natural-language processing, and automated code generation can support hands-free programming and improve accessibility for users with diverse needs.

Compared with previous voice programming systems that required users to speak programming syntax directly, CodeByVoice enables users to describe programming tasks using natural language and automatically converts these instructions into executable Python code. The system therefore reduces the cognitive burden associated with remembering strict programming syntax and provides a more intuitive programming experience.

Conclusion

The paper had suggested and created CodeByVoice, a voice-activated Python programming assistant, which allows users, especially people with disabilities to interact with programming environments through natural language voice interaction. Its inherent goals were enabling a dependable speech-recognition module, syntacticizing the spoken instructions into runnable Python code, running programs with both visual and audio feedback and allowing the user to edit, maintain and save code files using voice-interaction. Its results showed that the system was able to translate spoken commands into executable Python programs, allow simple program programming structures and display output in a graphical and text-to-speech format. The results of these experiments show that it is possible to combine speech recognition, command interpretation and code generation in one interface and develop a functional and convenient programming workflow that reduces the need to use traditional keyboard-based code writing habits. In addition to the technical implementation, the study also advances the overall research on the accessible computing and human-computer interaction area by showing how accessible programming environments may be made as multimodal interfaces including voice input, visual code representation and auditory feedback. However, the research is vulnerable to a number of limitations. The existing prototype focuses on simple programmable constructions and is dependent on how precise the current speech-recognition systems are and can still give some errors to interpreting complex commands or diverse speech patterns. The system could be im-proved in the future by adding more sophisticated natural language processing features, expanded support of more advanced pro-gramming tasks and testing it with users who have a wide range of interface accessibility requirements. These developments would be supportive of the functional usability of voice-based programming software. On the whole, this study is just a first step to more inclusive programming tools and a better understanding of how voice-controlled technologies can alter software development is possible.

References

[1] Begel, A., & Graham, S. L. (2005). Spoken programs. Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing. [2] Pane, J. F., & Myers, B. A. (2001). Usability issues in the design of novice programming systems. IBM Systems Journal, 40(2), 529–549. [3] Van Brummelen, J., Weng, K., Lin, P., & Yeo, C. (2020). Conversational programming systems and machine learning interface design. [4] Lucas Rosenblatt. 2017. VocalIDE: An IDE for Programming via Speech Recognition. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS \'17). Association for Computing Machinery, New York, NY, USA, 417–418. https://doi.org/10.1145/3132525.3134824 [5] Mankoff, J., Hayes, G., & Kasnitz, D. (2010). Disability studies as a source of critical inquiry for the field of assistive technology. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility [6] Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., & Kingsbury, B. (2012). Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine. [7] V. Kumar, “A next-generation AI voice assistant for computer: Multilingual, secure, and context-aware interaction,” Asian Conference on Communication and Networks, 2025. [8] S. Revankar, S. Deshpande, A. Sayeed, and A. Tandale, “Sanvaad: A multimodal accessibility framework for sign language recognition and voice interaction,” arXiv preprint, 2025. [9] B. H. Juang, “Speech recognition and acoustic signal processing advances,” IEEE Signal Processing Magazine, vol. 39, no. 2, pp. 10-20, 2022. [10] D. Povey et al., “The Kaldi speech recognition toolkit,” IEEE Signal Processing Society, 2011. [11] Apple Machine Learning Research, “Improved speech recognition for people who stutter,” Apple Research, 2023. [12] M. Bond, “Spoken AAC: Assistive communication application for speech-impaired users,” Spoken Inc., 2025. [13] S. Lee et al., “Intelligent personal assistant implementing voice commands using speech recognition,” IEEE Conference Publication, 2020.

Copyright

Copyright © 2026 Epili Santosh Kumar Patro , D. Shiva Naik. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.